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ABSTRACT 



In 1994, the Minister for Education in Northern Ireland 
announced a major school improvement initiative for Belfast, Raising School 
Standards (RSS) . Aspects pertaining to the management of this school 
improvement initiative, including the launch and selection of 14 project 
schools, are examined here. The paper analyzes the process of action planning 
in schools and addresses three of the initiative's goals: the quality of 
school management; teaching and learning; and educational standards. 

Described in detail are the external evaluation of the RSS initiative; the 
launching of the initiative; the selection of the schools, which included 
secondary and primary schools; schools' reactions to being included in RSS; 
the schools' action plans and the elements within the action plan; 
implementing and improving the action plans; the impact of RSS as indicated 
by baseline measures, internal evaluations, and questionnaire data; the 
impact of RSS on school management; areas in which RSS appears to have had 
the least impact; the impact of RSS on teaching and learning, including 
curriculum development and teachers' focus on learning; the impact of RSS on 
secondary school examination results and teachers' focus on pupil outcomes; 
and the legacy of RSS and challenges for the future. Conclusions reveal mixed 
findings. (Contains 64 references.) (RJM) 
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Concern about the quality of schooling in disadvantaged urban areas has been especially marked 
throughout the U.K. during the 1990s. This led, in part, to the then Minister for Education in 
Northern Ireland, announcing in May 1994 a major school improvement initiative under the Making 
Belfast Work (MBW) programme. The Raising School Standards (RSS) initiative was intended to 
help schools address significant disadvantage and under achievement amongst their pupils. The 
project was designed to target a small number of secondary schools and their main contributory 
primary schools. Additional funding of £3m, over the three year period was allocated. A major 
difference between this initiative and other school improvement projects was the substantial financial 
investment agreed over a three year period and the strong emphasis placed specifically on pupil 
outcomes. 



In evaluating the MBW RSS initiative it is important to acknowledge the context of civil unrest which 
has been an everyday fact of life for people living in the city for over a generation. Recent political 
initiatives to move forward the peace process have been welcomed by all who are concerned about 
the quality of life in Belfast, although uncertainty about the future remains evident. Whilst outside 
there has been great tragedy, the schools themselves have been seen by many as "peace havens" (BELB 
advisor 1996) although the impact of "the Troubles" has been seen to have had a direct effect on some 
students' behaviour. 

During the last fifteen years there has been a growth in public and policy concern about educational 
standards, particularly in literacy and numeracy, and widespread recognition of the importance of 
raising standards to meet the increasingly complex economic and social needs of society in the 21st 
century. This, in turn, has led to the demand for greater accountability of schools and teachers from 
policy makers, inspector and parents. Adding to the debate has been increasing evidence from school 
effectiveness research (Rutter et al 1979, Reynolds, 1985, Mortimore et al 1988, Sammons et al 1994) 
and more recently school improvement (Hopkins et al 1994, Myers K 1995, Stoll and Fink 1996, 
Barber and Dann 1996, Sammons et al 1997) about what makes an effective school. This research 
evidence has highlighted the existence of 1 1 key characteristics of effective schools (Sammons et al 
1995). 



Since the introduction of the Educational Reform Act (ERA) in 1988 in the UK, schools have 
experienced a period of rapid change (the introduction of the National Curriculum, LMS [1992 N.I.], 
School Development Planning, Appraisal, statutory testing at the end of Key Stages etc.). The 
statutory publication of examination results in 1992 (Northern Ireland) resulting in league table 
rankings in the press added yet another dimension to the debate about standards. The argument as 
to whether raw results tell us more about the catchments served by schools rather than the quality 
of education in schools in socio-economically disadvantaged areas, has remained a source of 
controversy (Mortimore, Sammons and Thomas 1994). 

The school effectiveness research base provides evidence of the need to develop value added 
approaches for the evaluation of school performance. Such approaches are becoming increasingly 
widespread and a number of Local Education Authorities have introduced innovative schemes (for 
a review see Thomas and Sammons 1997). The School Curriculum and Assessment Authority (SCAA 
now QCA) has undertaken work in an attempt to develop value added approaches for the reporting 
of schools' national assessment results at Key Stage (KS) 2 and KS 3 in England and Wales, while in 
the Northern Ireland context the Department of Education Northern Ireland (DENI) has funded 
research to develop a comprehensive value added framework for the evaluation of school performance 
as part of the expanded Raising School Standards Initiative. 

In evaluating the success of the original MBW RSS initiative it is important to remember the highly 
selective nature of secondary education in Northern Ireland as well as the high levels of 
unemployment and socio-economic disadvantage evident amongst many schools in Belfast. Pupil 
choice at the secondary phase in particular is complicated by; selection by apptitude (11+), 
geographical boundaries and religious background. It is known that there is a relationship between 
the concentration of socio-economically disadvantaged pupils (those eligible for free school meals and 
those with low attainment at entry) and general levels of pupil achievement. Contextual effects on 
secondary school pupils' achievement have been identified by a number of school effectiveness 
researchers (Goldstein et al 1993, Willms 1986, Sammons et al 1994, 1997). Inevitably, a selective 
educational system is likely to exagerate such contextual effects amongst the least popular/ and most 
disadvantaged secondary schools. 

Making Belfast Work 

The funder of the initiative, Making Belfast Work (MBW), a programme within the Department of 
the Environment, was launched in July 1988 to "strengthen and target more effectively the efforts 
being made by the community, the private sector and the Government in addressing the economic, 
educational, social health and environmental problems facing people living in the most disadvantaged 
areas of Belfast" (March 1995: MBW Strategy Statement). As MBW is concerned with urban 
regeneration it has an important role in the delivery of the Government's Targeting Social Need 
(TSN) strategy designed to provide financial assistance to overcome social disadvantage. 

A recent paper by MBW (1997) made the following comments, "education is pivotal to the Making 
Belfast Work (MBW) strategy for urban regeneration within the city of Belfast. MBW's focus on 
education reflects one of its core aims to 'improve the ability of residents to compete for jobs' and is 
a considered response to a long standing awareness of the link between socio-economic disadvantage 
and educational attainment." Their intervention is focused upon their own core wards, and 
recognises the importance of including all parties; The Department of Education, the Education 
Library Boards, the Council for Catholic Maintained Schools, and the individual schools and parents, 
in the process of improving performance. That their focus on improving attainment at primary and 



secondary level is set clearly in the context of other social objectives reflects their holistic approach 
to the problems (Hutson forthcoming 1998). 

The Aim of the Project 

The project focused on four secondary schools (two maintained Catholic and two controlled 
Protestant) identified as having high levels of under achievement and their ten main contributory 
primary schools. The primary aim of the project was to provide additional support and resources to 
schools with the overall objective of accelerating an improvement in the performance and 
employability of school leavers by:- 

1 improving where appropriate the quality of management, teaching and learning 
within the school; 

2 improving the standards of literacy and numeracy and overall standards at both 
primary and secondary level; 

3 improving the level of qualifications which they achieve at school and improving access to 
further education; 

4 improving links with local industry 

5 increasing parental involvement 

6 dealing with problems of discipline in class 

7 improving attendance and punctuality 

It should be noted that of these seven foci, three contain measurable pupil outcomes (2, 3 and 7) 
whereas two focus on school and classroom processes which influence the quality of the educational 
experience (1 and 6). By contrast, 4 and 5 concern aspects of links with the broader community. 



THE EXTERNAL EVALUATION OF THE MBW RSS INITIATIVE 

In May 1995 the Belfast Education and Library Board (BELB) commissioned the International School 
Effectiveness and Improvement Centre (ISEIC) at the Institute of Education, University of London 
to undertake an external evaluation of the initiative. As well as an investigation of the impact of the 
overall project it was thought desirable to examine the experience of implementing the project in 
individual schools in order to identify the factors which facilitate improvement and barriers to 
success. The evaluation was also intended to examine the general implications of the project for 
school improvement projects elsewhere and to identify examples of good practice for dissemination 
to other schools. 

Individual schools involved in the RSS initiative were required to submit Action Plans for approval 
by the Central Management Committee (CMC) demonstrating how they proposed to meet the stated 
aims and objectives of the initiative. Schools were also required to engage in self-evaluation and to 
collect relevant information to enable BELB to review the impact of the initiative, including the 



identification of a range of targets and baseline measures. A key feature of the initiative was to be 
focus on the use of measures of pupil achievement to establish the extent to which improvements in 
standards occur. 



A number of commentators have pointed to the value of a case study approaches to increase 
understanding of the processes which foster school effectiveness and positive change and provide the 
"rich description" of processes needed by practitioners concerned with school improvement. The 
National Commission on Education's (1996) Success Against the Odds study of 11 schools in 
disadvantaged areas provides a source of evidence relevant to policy makers and practitioners 
concerned with school improvement. Other school effectiveness research, which combined detailed 
qualitative case studies of both more and less effective secondary schools and departments with 
quantitative value added analyses of academic effectiveness at GCSE, of relevance to the evaluation 
of the project is provided by Sammons, Thomas & Mortimore (1997) and Smyth (1998). 

The results of research on the impact of School Development Planning (MacGilchrist et al, 1995) also 
has implications for the evaluation of the project. This drew particular attention to the need to 
examine the extent to which activities included in schools' written plans actually affect classroom 
practice and the extent of staff ownership of, and involvement in, preparing the plan. 

The extended MBW RSS evaluation adopted a case study approach in considering the four secondary 
and ten feeder primaries involved in the BELB project. The case studies recognised both the overall 
aims of the initiative and the differing aims of individual schools expressed in their individual Action 
Plans. A variety of sources of data and evidence were considered:- 

a) document analysis 

school and authority (BELB and DENI) level e.g. school development plans, RSS Action 
Plans, school policy documents, Inspection reports 

b) schools' statistics 

pupil outcome data, attendance, expulsions and suspensions as well spending patterns of the 
additional resources 

c) interviews to explore the views of key personnel 

BELB advisors, Principals, Co-ordinators, Field Officers and Chairs of Governors (COGs) 

d) questionnaires to explore perceptions of the school 

pupils/parents (Primary = Year 5 and 7, Secondary Year 9 and 11) and all teachers 

This paper examines a number of aspects pertaining to the management of a school improvemet 
initiative, including the launch and selection of schools. It also analyses the process of action pl annin g 
in schools and addresses three of the main aims of the overall initiative: the quality of school 
management; teaching and learning (1) and educational standards (2 & 3). Barriers to the 
implementation of the project and factors which facilitated its success are identified. The evaluation 
was commissioned to explore the impact of an educational initiative and the factors which influenced 
its implementation and success. It does not attempt to evaluate the success of individual schools. For 
full details of the evaluation see Taggart & Sammons, 1998. 
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Launching the Initiative 



It was reported that the idea for the initiative came originally from DENI, set against a backdrop of 
concerns about under achievement in the Province as a whole. Interviewees said it took some two 
years from inception to the press announcement by Education Minister, Michael Ancram MP on 1 1th 
May 1994. RSS was one of five major educational projects launched that year. 

How any project is launched is important in terms of setting the tone for what is to come. The 
Minister's press release referred to schools "who are presently achieving no or only very poor 
qualifications" but did not publicly name those schools. The press coverage (Ulster Newsletter and 
Belfast Telegraph) factually reported the main points from the press release. The Irish News (12.5.94) 
however published names of the four secondary schools and called them "schools identified as having 
the highest levels of under-achievement." This high profile public naming was not well received by 
school staff who were mindful of their institution's reputation in the community. 

• Before any public announcement a great deal of work has to be done with schools to ensure they are 
prepared for critical publicity. 

Clearly, the same issues have been debated elsewhere in the UK concerning the relative merits of the 
popularly coined 'naming and shaming' policy concerning failing schools. Public naming is seen by 
some to be a necessary stimulus for improvement for schools with serious weaknesses, but others 
believe it can exacerbate schools' difficulties, lowering staff morale, making teacher recruitment 
harder and increasing problems due to falling rolls (Mortimore and Whitty 1997). 

• The approaches made to schools to invite them to participate in an initiative must he given careful 
consideration and procedures must he uniform. It is important that schools are given sufficient 
notice of inclusion in any developmental project work to enable them to think through the 
implications of this with their Chair of Governors and key personnel in school. 

The reaction to being included 

Despite the controversy generated by the launch, only two CoGs thought their school reacted badly 
from being included in the initiative. In one school the CoG perceived the staff were "alarmed" about 
inclusion as some stigma might be attached to the school. In the second school the CoG thought the 
Principal and staff were "apprehensive" because of the extra work the initiative would generate. All 
of the remaining CoGs thought their schools reacted very positively to being included in the 
initiative. One CoG of a secondary school said he thought the staff accepted the criteria for selection 
as there seemed "no point in arguing about it," and the additional resources would help teachers 
"tackle problems we knew existed." 

The reaction of the four secondary Principals to being included in the initiative was generally more 
negative than positive. In one school, it was reported that the senior management had to work hard 
to prevent a "negative feeling" from being labelled a failing school from prevailing. They deliberately 
presented RSS to the staff as "recognition of the advances we've made" with the funding being seen 
as "help to enable us to continue." The challenge for this Principal was to "turn it around": to foster 
a "no blame culture" so that staff would view the initiative positively. He reported some success in 
this area. A second Principal reported how betrayed and angry he and his staff were, feeling the 
school had being unfairly stigmatised. A third Principal thought staff saw RSS as just "another 
initiative" (one of many the school had been involved in). One Principal was unable to comment as 
he was not in post at the time. 



Whilst some secondary school Principals may have felt uneasy by press coverage of the initiative this 
did not affect their primary colleagues in the same say. Primary school Principals saw the criteria for 
selection (low achievement) being unique to secondary schools with feeder primaries chosen to 
support the four secondary schools. The most common reaction to inclusion in the initiative in the 
primary sector was excitement about the additional funding. This was not however all that primary 
Principals considered. In three schools there were concerns about work load issues. 

The majority of Principals disliked the title of the initiative because it suggested that their standards 
were too low. 

• If initiatives are sprung on teachers their reactions could delay progress in the initial stages and 
deflect their focus away from the aims of the project. 

Selection of Schools 



It was considered important when setting up the initiative that the selection of schools was based on 
objective criteria. In addition, a stated pre-requisite for inclusion, regardless of other factors, "was the 
commitment within the management team to achieve successful change." The final list of schools 
included in the initiative followed a 'catchment area' approach to identify feeder primaries for the 
four selected secondaries. Once schools were selected they were asked to sign a formal contract. 

Secondary Schools 

The initiative had a clear focus on raising school standards and the secondary schools included in the 
project were selected after consideration of DENI statistical information using the following criteria:- 

a performance index which took account of both the number and level of GCSE and other 
vocational qualifications; 

the percentage of pupils who achieved A - C in at least five subjects; and 
the percentage of pupils with no GCSEs. 

An AVOWQI (Average Weighted Qualification Index) was considered the most appropriate indicator 
for selection purposes. Points were assigned according to grades obtained by school leavers. The final 
AVOWQI was calculated by summing the total of grade points for all pupils in each school across 
the three year period 1989/90 - 1991/92 and dividing this by the total number of school leavers for 
that school across the same three year period. 

Primary Schools 

The primary schools invited to participate had to satisfy the following criteria:- 

the primary school must be a feeder for one of the participating secondary schools 

25% or more of P7 pupils must transfer to one of the participating secondary schools 

the number of pupils who transfer must constitute 10% or more of the total number 
transferring to one of the participating secondary schools and 



the results of transfer tests. 



All percentages were based on an average over the three year period 1989/90 - 1992/93. 

Although the selection criteria for inclusion in the MBW RSS initiative were clearly identified, it was 
inevitably (given the large amounts of money involved) closely scrutinised. Those interviewed 
reported some controversy about the selection of schools. Much of this debate ocused on the 
following issues:- 

i Targeting Social Need (TSN) 

ii selection of the four secondary schools in relation to under achievement 

iii selection of the feeder primary schools 

i Targeting Social Need 

The debate about the amounts of money allocated to any one individual school must be seem within 
the context of a much larger debate taking place in Northern Ireland at present about the efficacy of 
the "Targeting Social Need" (TSN) policy. 

In order to help combat 'social deprivation' and 'need' the policy of TSN was introduced as a 
Government Public Expenditure priority for all Northern Ireland Departments in 1991. The 
operation of the TSN policy in DENI since 1994, enabled five per cent 'top slicing' (approximately 
£40 million) of the education budget to be redistributed across the five Education and Library Boards. 
This money is additional to that distributed to schools by the normal Local Management of Schools 
(LMS) formula. The Boards distribute TSN money to schools using the number of children on role 
entitled to free schools meals as one indicator of social need. Some boards also include in their 
distribution formula to schools other indicators of disadvantage i.e. the special needs of pupils etc., 
which has resulted in some anomalies in the amounts individual schools receive in TSN allowances 
between the different Education and Library Boards. The Northern Ireland Affairs Committee, 
Second Report Feb. 1997 on Under achievement in Northern Ireland Secondary Schools came to the 
conclusion that "there were no clear mechanisms for ensuring an equitable allocation of resources to 
schools with similar proportions of deprived children in different Board Areas" (page xix). It would 
appear that the impact of the TSN policy in tackling under achievement has been little researched and 
the Affairs Committee concluded that "DENI should monitor the use of TSN money more closely" 
(page xxi ibid.). 

• To ensure adequate and equitable funding for social need and to avoid the problems of a 
geographical lottery may require consideration of how a common funding formulae could he used 
to target funds. The aim should he to meet the needs of the most disadvantaged pupils without 
rewarding schools with low standards. This would only he possible if information about pupil 
disadvantage and low performance were collected concerning pupil intakes to schools. Any 
formulae would need to he transparent and take account of existing knowledge of the relationships 
between social disadvantage and educational attainment. 

The TSN policy has a bearing on the initiative given that the funders, MBW, saw themselves as being 
at the forefront of the Government's TSN strategy. Critics of the initiative said they thought MBW's 
involvement with RSS ran counter to its remit in delivering TSN money, particularly in the selection 
of primary schools (which will be addressed later). It is clear that the use of TSN money is a matter 
of interest and concern for those working within the Northern Ireland educational system, not least 
those working in schools who are ever mindful of levels of resourcing in an increasingly competitive 
climate. 
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The situation in Northern Ireland is not different in this respect to that operating in England during 
this period. For example an analysis of LMS funding focusing on the definition and funding of 
"educational need" by Sammons (1993) produced very similar conclusions and indicated that a pupil 
with an identical score in terms of 'need characteristics' would attract different levels of funding in 
different Local Education Authorities (LE As). 

The links between poverty, social deprivation and under achievement continues to be a subject for 
debate amongst educationalists, employers and politicians (see Mortimore and Whitty, 1997, 
Robinson 1997, Sammons et al 1994; 1997). In this context it is relevant to note that, under the last 
Government, there was considerable reluctance to recognise any causal links between social 
disadvantage (as indicated by FSM) and educational achievement since this was felt to condone low 
expectations. Indeed methods of contexualising school performance data using such measures 
developed by OFSTED (Sammons et al 1994) were suppressed in 1995. The MBW RSS initiative was 
set up to help schools tackle under achievement, "aimed specifically at improving the levels of 

performance including those at secondary schools who are presently achieving no or only very 

poor qualifications." (Michael Ancram press release May 1994). It is therefore unsurprising that the 
secondary schools chosen served high proportions of pupils from socio-economically disadvantaged 
backgrounds, and thus scored highly on the free school meals indicator of social deprivation. 

Clearly the bracketing together of TSN and RSS did result in a "layer of confusion" but the criticism 
that the "GCSE criteria were never defined precisely" or that "they were not applied robustly," made 
in press coverage (McGill, 1996) cannot be levelled at the original Making Belfast Work RSS initiative 
on the evidence provided for the evaluation. 

• It is helpful when launching an initiative to consider the effects of the selection criteria on a wide 
audience. Where the criteria for selection is unambiguous it may be sufficient to publish this 
information only. Where data are more complex and could be misinterpreted then it is perhaps 
wise to include some exemplification material to assist in the interpretation of the selection criteria . 

The rationale behind the selection of primary schools was in keeping with the initiative's aims of 
improving levels of achievement in the secondary sector by placing particular emphasis on improving 
the basic literacy and numeracy levels of children entering those schools in Year 8. However, given 
the debate about TSN, the inclusion of some of the primary schools appeared anomalous. There were 
many other primary schools in Belfast with much higher levels of social deprivation and need, so the 
inclusion of some of the MBW RSS primary schools was seen as controversial in the MBW's stated 
policy of Targeted Social Need. Given that the stated criteria for selection was on transfer numbers 
(and Grade 4/ opt out) alone, however, it was inevitable that other aspects, such as low levels of 
academic standards and disadvantage could not be covered fully. 

To influence the levels of achievement in secondary schools it was important when setting the 
selection criteria that the actual numbers of children transferring from a given primary was 
considered. But in looking at the actual numbers rather than a percentage of Year 7 leavers, the 
selection criteria worked in the favour of large primary schools at the expense of smaller schools. 
Some smaller primary schools, not selected for RSS, sent a higher percentage of their pupils to RSS 
secondary schools. Some of the larger RSS primary schools whilst sending a larger number of actual 
children to one of the RSS secondaries could not count this as one of their more popular receiver 
schools in percentage terms. 

Thus although the selection criteria, devised for the MBW RSS project appeared straightforward this 
inevitable bias towards larger primary schools (the number had to equal 10% or more of the total 



number transferring to a selected secondary) added to the controversy and debate about the selection 
of the primary schools. Against this it can be argued that the selection of larger primaries ensured 
that larger numbers of children were covered by the project. 



# Any criteria for the selection of schools must be relevant and evidence-based, clearly explained and 
applied fairly. 



The implications of being labelled an underachieving school 

For BELB the choice of the secondaries was an acknowledgement that these schools were “faced with 
problems”, but given the publicity, being included on the initiative brought the public stigma of 
“failing”. Some Principals were uncomfortable with the qualifications based performance indicator 
used, which they felt left other areas of success in their schools unacknowledged. All of the four 
secondaries knew they scored poorly in terms of examination performance but staff would not 
necessarily have described their school as failing. It should be noted there was also a feeling amongst 
colleagues in non-selected schools with high levels of disadvantage that failure was being rewarded and 
that more able Principals were being penalised for “being effective managers who run successful 
schools." It is important to recognise that the RSS schools had the advantage of considerable 
additional resources and external support for school improvement as a result of the project. 
Nevertheless, the demands for accountability and the requirements for successful improvement 
strategies are not easily reconciled as experience elsewhere in the UK also demonstrates (TES Nov. 
1997 ). 



# The implications of tackling poor performance must be thought through very carefully. How this 
impinges on morale within a school and on public confidence outside of the school must be managed 
by the project leaders. 

# Schools need to have some time to adjust to being labelled and stigmatised as under performing. 
The Principals, staff and Governors of these schools need to be worked with sensitively “in order to 
get over the improvement message. ” 



THE SCHOOLS' ACTION PLANS - WRITING, IMPLEMENTING AND EVALUATING 

One of the key features of the MBW RSS was that selected schools had to submit for approval to the 
Central Management Committee (CMC) an “Action Plan” (AP) focusing on a whole school approach 
to meeting the objectives of the initiative. In many respects the AP was intended to be the school's 
blueprint for improvement. As the initiative was to extend over three years, it was considered 
important that schools build in improvements to their structure and working arrangements so they 
could maintain and further improve standards within the normal LMS funding arrangements after 
the conclusion of the project. Schools also had to seek approval from their Board of Governors who 
were asked to give undertakings to assist in the monitoring and evaluation of the programme. 

The APs were designed to broadly detail the problems which schools wished to tackle and the 
methods they would use to do so and in addition, schools were advised that the plans should identify: 



a) what can be achieved within existing resources, including funds to which schools are entitled 
under the LMS formula 

b) funds from other MBW projects and from the Belfast Action teams (schools already 
participating in MBW initiatives such as the Numeracy/Literacy project and the Discipline 
project were expected to indicate what benefits they had already derived from their 
participation in such projects, how lessons learned would be disseminated throughout the 
school and how ongoing work would be subsumed within the overall school plan) and 

c) additional resources needed from the project. 

All APs had to be agreed by the CMC and any additional funding was to be determined on their 
judgements of the merits of each individual proposal. 

Before considering the place of Action Planning within MBW RSS, Action Planning as a process need 
some explanation. The most common form of Action Planning met by schools would be following 
an inspection. In the English context all schools inspected must produce an Action Plan within forty 
days of receipt of the Inspection Report. The importance of the Action Plan, guidance on its 
production and the link with school development plans is outlined in DfE Circular 7/93. The process 
of inspection as described in DENI's The Inspection Process in Schools (1992) and in Evaluating Schools 
(1992) does not place the same emphasis on the production of a school's Action Plan. “Within four 
months of the date of publication of the report, the Board of Governors is required to submit to the 
Department a written response outlining the action that has been taken, or planned, to address the 
issues in the report." 

It is widely recognised that school improvement needs to be planned, it is not therefore unreasonable 
to suggest that schools might be more confident in approaching Action Planning if they had 
experience of planning for school improvement. The process by which most schools would be 
familiar with planning for improvement would be through the production of a school development 
plan (SDP). There is no statutory requirements for a school to have a development plan in the UK. 
However given the evolution of school development planning (MacGilchrist et al, 1995; Hargreaves 
& Hopkins 1991; 1994] the presence of a plan has become increasingly important as a developmental 
tool. 

In England, schools have been assisted in formulating SDP by a number of Government publications 
(DES 1989, 1989b, 1990, 1991 and OFSTED 1995). Mention of school development planning in the 
NI context is contained in The Inspection Process in Schools (1992) DENI. It states that "Increasing use 
is being made in schools of Development Plans, in which schools outline their policies, identify aims, 
undertake their own evaluation of how well or how far those policies, aims and objectives are being 
implemented, and identify priorities and targets for review and development" (p 7) and in Evaluating 
Schools (1992) comments under "Management Arrangements. Organisation of the curriculum" that 
the organisation of the curriculum can be considered good when: the senior management and the 
teachers have contributed to discussions of the place of the statutory curriculum within the wider 
curriculum and have agreed on a strategy for its implementation; there is a costed Development Plan 
covering curricular needs, changes and potential improvements; this is available to all interested 
parties, including parents" (p 9). Hargreaves and Hopkins (1991) argue that leadership, management 
and efficiency should be judged by the extent to which, "the school, through its development 
planning, identifies relevant priorities and targets, takes the necessary action, and monitors and 
evaluates its progress towards them" (p21). Matthews and Smith (1995) state that "the introduction 



of the inspection system, together with post-inspection action planning, is making a contribution to 
school improvement" (p 29). 

Thus the reliance placed by the MBW RSS initiative on Action Planning and school development 
planning should be seen in the wider context of the highest profile given to school inspection and 
reflect the importance attached to planning as a vehicle for improvement by the Inspection service 
in both England and NI at this time. 

* If new and innovatory techniques are to be part of a project 's methodology, the implications of this 
for school personnel must be considered before implementation. Where this is likely to present 
challenges, project leaders should ensure that practical measure are put into place to help overcome 

difficulties. 

• If Action Planning is to be used as a tool for school development, those responsible for this task must 
be equipped to manage the process. They need to be able to build on existing school development 
frameworks and set realistic targets. 

Elements to be considered within the Action Plan 

The MBW RSS APs were intended to be specific to the needs of individual schools. Nonetheless, the 
programme was expected to include some or all of the following elements:- 

developing effective management at all levels within the school 
the development of literacy/ numeracy across the school 
raising expectations amongst pupils, parents and teachers 
increasing educational achievements 
involving parents in the support of schools provision 
improving attendance and punctuality 

increasing and improving links between the school and employers 
developing appropriate certifiable vocational courses 

developing and implementing effective procedures to monitor and evaluate the curricular and 
organisational feature of the school 
preparing and planning programmes. 



A proforma to help schools write up their APs was developed containing, in the first year, the 
following headings:- 

i curriculum focus 

ii year groups to be involved 

iii current position 

iv targets 

v time-scale 

vi staff involvement 

vii resources required - including costings and 

viiii evaluation. 

Schools were asked to take into account their existing priorities, as set out in the school development 
plan for 1994-95, when completing these. 



Areas covered by Action Plans 

Having considered the APs the CMC identified the following areas across all schools:- 

i literacy 

ii numeracy 

iii pastoral care/ personal and social education 

iv closely linked with in was the development of positive discipline and rewards systems 

v parental involvement 

vi closer links between primary and secondary schools and 

vii staff development 

The BELB perspective on Action Planning 

The majority of BELB Advisors interviewed considered the production of an AP to be the main 
challenge for schools in the early stages of the initiative and many of the problems associated with 
the first year of the initiative were seen as a result of a lack of experience in Action Planning. One 
Field Officer summed up the views of many when reporting that because Action Planning was 
couched in terms such as "success criteria," "target setting," "areas of planning," it demanded from 
Principals a clarity of thinking" which took some time to develop. 

BELB Advisors were familiar with the processes of development planning but they reported facing 
some specific challenges when considering the whole notion of Action Planning. Some Advisors 
considered themselves, "ill prepared for what APs was all about in terms of the depth, detail and 
rigour of what was required. " 

The submission of the first APs was considered by all to be very rushed. The original timetable 
identified January 1994 as the time when schools were to be approached and invited to become 
involved in the initiative. Implementation was to begin in April 1994. In the event, this time scale 
proved impossible to adhere to because the initiative was not officially announced until May 1994. 
As the Northern Ireland academic year finishes at the end of June, this left little time in which to 
begin any necessary consultation. Most schools began writing their plans when they returned to 
school in the Autumn of 1994. By November 1994 all schools had submitted their Action Plans to 
the CMC. As a consequence the initiative did not start in earnest for some schools until the 
following January (1995), almost three quarters of the way through what was meant to be the first 
year of the initiative. 

• The time-table of a project needs to be carefully mapped out. Any time-table needs to be 
sympathetic to the other calendar demands on schools. Schools need time sufficient time to plan for 
developments, before embarking on implementation. Time-tables need to be explicit and realistic 



The lack of expertise, coupled with the very short time in which schools were asked to formulate and 
present plans, resulted in APs being submitted which BELB Advisors could see would be problematic 
when it came to implementation. The most commonly reported problem in the first APs was their 
tendency to be over-ambitious. Where some schools identified strategies for improvements in all of 
the nine key areas, Advisors had serious doubts about the extent to which the schools could deliver 
all the improvements they were aiming for simultaneously. Despite the production of an AP 
proforma, there was in practice no common method of completing this by the schools. 
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The analysis of schools' APs showed little consistency in how an AP could be approached in the first 
year. The evaluation mechanisms in particular showed wide variations and targets were often 
non-specific. It is clear that at the start of the MBW RSS no consistent approach was adopted for 
Action Planning. This had important consequences for the later evaluation of the impact of RSS (for 
details see Sammons & Taggart, 1998). 



• Where new techniques are introduced into schools it can be helpful to develop exemplification 
material for training purposes (to be used before implementation). The use of such material would 
encourage consistency and specific key areas i.e. evaluation mechanisms, could be highlighted. 



Implementing the Action Plan 

A number of factors helped Co-ordinators implement their Action Plans. The most significant 
factors were: 

• having the co-operation of staff 

• linking the school's staff development programme to the Plan 

• the monitoring and evaluation of target setting 

• being given the opportunity to keep staff informed of developments through regular meetings 
work at the very beginning of the project which had an immediate impact on the school e.g. 
the redecoration and re-equipping of rooms for library and reading club, the introduction of 
new books and resources etc. 

• the courses run by BELB 

Other factors mentioned as important in some schools were: the arrival of a new Principal, having 
a non-class teaching Co-ordinator, the re-deployment of staff and focusing on existing expertise on 
the staff. 

Two groups of factors made Action Plan implementation more difficult. The first was concerned with 
staffing and personnel issues and the second with the central organisation of the initiative. 

Staffing issues included: 

• staff with entrenched attitudes 

• poor departmental leadership 

• not having the right people in post at the right time 

• the Co-ordinator feeling isolated 

Central organisations issues included: 

the time scale for developments being too rushed at the outset 

the time scale between the submission of AP and the arrival of funding being too long 
complications in the system for ordering resources 
not enough support for staff development 
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Improving the quality of Action Plans 



It was recognised by BELB that the quality of the first APs were often poor, given the short time scale 
and lack of expertise and experience in school . The poor quality of the initial AP coupled with the 
emphasis on resources and the budget cycle were thought to have sent mixed messages to schools. 
Schools were advised not to focus on financial matters but then felt pressured to spend large amounts 
of money very quickly without time for adequate reflection and planning in order to fit in with the 
demands of the financial year (1.4.95.) for first year's APs. This meant that the initiative was 
perceived, in the early stages (possibly the first six to nine months) as being "resources driven." 

In response to these difficulties BELB personnel began to look at other school improvement 
initiatives, other school improvement networks and to recognise that new strategies needed to be put 
in place if the initiative was to be refocused on its underlying aims. This led to a conference (9th and 
10th May 1995) in which Professor David Reynolds was a key note speaker. Professor Reynold's 
input at the Belfast conference attended by Principals, Co-ordinators, DENI and BELB gave a general 
background to the "school effectiveness movement" in both America and England. He focused on 
questions of urban disadvantage and school effectiveness and improvement. There was a particlar 
stress on: academic achievement; reward based control systems with reference to areas of consistency, 
cohesion and constancy; and the role of the head teacher. In relation to School Development in 
Raising School Standards the emphasis was on: the means of achieving outcomes; organisational 
change and the change culture; altering behaviour; the focus on teaching and learning; and the 
development cycle 

After this keynote conference delegates were asked to get involved in workshop sessions which 
explore practical steps towards improvement. They also had the opportunity to analyse a practical 
example of an Action Plan teasing out from this example the key factors which could assist school 
improvement. The July conference was reported by many of the interviewees to be a major turning 
point in the development of the MBW RSS initiative. The subsequent evaluations of the day (by 
BELB) suggest that the key note speech and the input on Action Planning were the most valuable, 
with delegates finding Professor Reynold's contribution particularly thought provoking. After the 
conference, Principals and Co-ordinators assisted by BELB personnel felt they were able to apply 
much of what they had heard (and done) on that day to the situation in their own school. They were 
able to look at their APs in a new light and to set this into the overall context of school improvement. 

• The involvement of an outside consultant can be crucial in the school development process, whether 

as a key-note speakers ' or 'critical friends The ability to identify the most appropriate outside 
consultant is dependent on opportunities for senior personnel to network with colleagues in the 

field. 

This subsequent emphasis on developing schools' capabilities for Action Planning meant that schools 
were more involved in the monitoring and evaluative process which, "brought a new dimension to 
their schools, which is significant." (BELB Advisor) This affected planning in almost every school, 
giving it a much sharper edge. At the start of the initiative, where schools had thought about school 
improvement they appeared to have visions, but lacked any sense of the detail needed to make them 
achievable. The conference was reported to have helped Principals, Co-ordinators and Advisors 
identify the important small steps which would help lead towards improvement. 

Using Action Planning as an "effective management tool" was also seen to have enabled Advisors to 
work with Principals and Co-ordinators "on monitoring and evaluation and how you tie in finances 
and resources to your objectives." Whereas schools were thought to be very weak at this before the 



initiative, subsequent training and guidance was felt to have done much to help them to improve this 
area of their work significantly. The approach to Action Planning taken after the first year appeared 
to help Principals and Co-ordinators narrow the focus of their APs and make their improvement 
programme much more realistic and manageable. Advisors encouraged schools to focus on one or two 
of their key objectives so that change could go beyond the cosmetic. 

Schools' Internal Evaluation of the Impact of MBW RSS 

The evaluation mechanisms used to determine the success or otherwise of the initiative were 
inextricably linked to the schools' APs. Evaluation does not exist in a vacuum. It should not be seen 
a bolt on appendage but part and parcel of the planning cycle (MacGilchrist et al 1994, Barber and 
Dann 1995, Sammons et al 1997). The Board and Schools' own capacity to evaluate of the initiative 
improved markedly after the first year as schools became more adept at producing annual APs which 
showed a greater focus on the aims of RSS. 

Although schools were given suggestions as to the type of monitoring and evaluating procedures they 
could engage for the initiative, this was not made a requirement at the start of the project. Advisors 
reported that some baseline measures were in place for evaluation purposes but this was inconsistent 
across the fourteen schools and practice in this area showed a great deal of variation. The 
mechanisms for evaluation identified in the APs ranged from some quantifiable data such as 
monitoring statistics, to more subjective perceptions of "how things were going". 

Clearly some areas of the initiative are much easier to evaluate than others. In the core curriculum 
subjects of English language and mathematics pupil progress can, and was, in some schools explored 
by analysing gains in pupils' standardised test scores, athough not all schools adopted this strategy. 

An overall lack of data on baseline measures appears to have seriously weakened most schools' ability 
to take meaningful measures. As noted by OFSTED, "the absence of success criteria in some 
initiatives makes it difficult for participants to know and understand the impact of their work and 
means that they depend on subjective judgement or anecdotal evidence, neither of which are likely 
to be accurate about the past or a good guide to future planning" (p36). The first year of the MBW 
initiative clearly reveals this as a problem in many schools. Subsequently schools' abilities to APs 
improved markedly in most cases (for example, see Sammons & Taggart, 1998). 



THE IMPACT OF MAKING BELFAST WORK : RAISING SCHOOL STANDARDS 

The introduction to this evaluation acknowledged the particular difficulties faced by schools in areas 
of civil unrest. The National Commission on Education (1996) acknowledged that any school which 
attempts improvements is influenced by factors which operate from outside as well as inside the 
school. Whilst none of the case study schools in Success Against the Odds experienced the particular 
circumstances of life in Belfast, they were nevertheless, in difficult and challenging environments. 
The Commission reported that what marked these eleven schools out was their, "will to succeed was 
strengthened, rather than weakened, by calamities." (p 315). These were schools who were able to 
"strengthen and hone their vision in relation to whatever vicissitudes are thrown up by their 
environment." Working in Belfast is undoubtedly very challenging and school improvement here, 
more than perhaps anywhere else in the United Kingdom, has to be seen in terms of the extent to 
which "challenges have been turned into opportunities..." (ibid). 
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The primary purpose of the RSS initiative was to tackle under-achievement and by considering 'pupil 
outcomes,' to assist teachers to raise standards and improve the quality of pupils' educational 
experience. When considering 'impact' the external evaluation addressed not only 'pupil outcomes' 
but changes in other areas identified in the original proposal as important to the project (management, 
the quality of teaching and learning and eduational standards in literacy and numeracy). 



Baseline measures 

When the DENI Raising School Standards Initiative (expanded) was introduced in 1995 considerable 
time was spent establishing baseline measures. The establishment of baseline measures can assist in 
the monitoring, evaluation and management of any new development. They can be used as the basis 
for the development of 'success criteria' and enable issues of impact to be examined critically and 
objectively for example, baseline measures have been used to provide value-added feedback to primary 
schools (e.g. Sammons and Smees 1997, Strand 1997, Yang and Goldstein 1997,) and at secondary level 
(Thomas and Mortimore 1996). There use has been demonstrated by the Improving School 
Effectiveness project in Scotland (Robertson & Sammons, 1997) as part of a research initiative and 
development explicitly designed to link school effectiveness and improvement approaches. 

Although baseline measures were advocated at the start of MBW RSS (in 1994), schools did not appear 
to have received clear guidance on which to use and there was little consistency in approach across 
the schools as a consequence. 



The evaluation data used to examine the initiative's impact were collected during the second and third 
year of the initiative (October 1996 - July 1997). Development continued in schools after the data 
collection period and schools were encouraged to engage in exit planning via special meetings with 
BELB personnel. For some areas of the initiative it has been possible to demonstrate clearly, on 
standardised measures, the impact of the initiative in the short term. These tend to be curriculum 
areas which have established assessment practices such as reading/ maths tests. For other areas the 
evidence is more subjective reflecting participants' perceptions and in some instances may not be 
calculable for several years to come. Given the length of the initiative and the scope of the external 
evaluation, this paper cannot ascertain any long terms effect RSS might have. To explore this a 
follow-up study would be needed 2-3 years after the end of the project. 

The use of the questionnaire data 

As part of the evaluation, questionnaires were administered to three key groups : parents, pupils and 
teachers. The questionnaires have been used in two specific ways: 

1 to inform the external evaluation 

In order to do this the quantitative and qualitative responses were aggregated together across the 
fourteen schools to give a picture of the initiative as a whole. It is important to note that the sample 
for this survey was small, using only fourteen schools. The schools themselves were not randomly 
chosen but fitted a specific criteria for selection on the initiative. Therefore, these data cannot be 
used to make wider claims about the performance of Belfast schools as a whole. 

2 to inform individual schools 

Outside of the remit of the research proposal, each Principal was sent a report analysing their own 
school's information. This aggregated responses from all fourteen schools set beside the information 
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from their own school. The report was sent during September 1997, after the official end of the 
initiative in order to avoid any research bias or interference. 

The feedback questionnaire analysis was used to identify strengths and possible areas for further 
development. It was also able to reveal any significant differences between perceptions in individual 
schools and in the RSS sample as a whole. The approaches used, and the cut-off points for 
determining strengths, represent particular ways of looking at the data and it was suggested that 
schools could also use different approaches. Schools were given guidelines on interpreting their data 
and on the ways they might address some of the issues raised by the questionnaires. 



Reading and using the questionnaire data 

Before considering the following sections the methodology used to interpret the questionnaire data 
needs some explanation. In each of the questionnaires respondents were asked questions which 
related to the aims of the initiative. The pupil questionnaires were similar in that secondary pupils 
were asked whether certain things happened in their school: always - usually - hardly ever - and never, 
whereas primary pupils were asked if these occurred: always - most of the time - sometimes - never. 
Parents were also asked to respond to a range of statements as to whether they: strongly agreed - 
agreed - disagreed - strongly disagreed with them. 

The purpose of the pupils' and parents' questionnaires was to explore perceptions of their school in 
relation to the developmental areas of the initiative. The teacher questionnaire was different from the 
others as it sought to explore specifically the impact of the RSS initiative. In this questionnaire 
teachers were asked to respond to questions measured on two scales: 

Scale A: the extent to which the school had made improvements during the last three years. 

This was graded - significant improvement - slight improvement - the same (no change) 
- slightly worse - significantly worse 

Scale B: the extent to which RSS had affected the above improvements. This was graded - 

significant effect - some effect - no effect. 

THE IMPACT ON SCHOOL MANAGEMENT 

The importance of leadership and management which can help to enhance a shared vision for 
improvement has been highlighted in much of the literature on school effectiveness and school 
improvement both in Britain and internationally (Gray 1990, Mortimore et al (1996), Stoll and Fink 
1996, Teddlie and Stringfield 1993, Hallinger 1996, Sammons et al 1997). The starting point for 
exploring the impact which RSS had on management was the analysis of the teachers' questionnaire 
which sought views on the extent to which teachers (in all fourteen schools), thought their school had 
improved over the period of the initiative for a range of management indicators. This was compared 
with the extent to which teachers thought RSS had influenced these improvements. Both the 
additional written responses on the questionnaires and information taken from interviews were used 
in interpreting the quantitative data. 

The questionnaire data was divided into two parts: 

i areas in which teachers considered RSS had a significant effect 

ii areas in which teachers considered RSS had no effect. 
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The tables below display the results for items rated as the most and those rates as the least affected. 



i areas in which teachers considered RSS had a significant effect 



% significant 


% no 


n = 189 


% improvement 


effect 


effect 






40.0 


17.2 


staff contributing to the school development plan 


74.1 


38.3 


10.4 


clear vision of what the school is trying to achieve 


92.4 


34.8 


22.5 


the staff development programme 


72.8 


ii areas 


in which teachers considered RSS had least effect 




significant 


no 


n = 189 


% improvement 


effect 


effect 






22.0 


26.4 


communications between SMT and staff 


69.7 


18.2 


33.0 


the quality of SMT decisions 


57.4 


17.9 


33.5 


staff cohesiveness 


53.3 


27.4 


34.1 


staff morale 


50.8 


23.7 


36.2 


the leadership of the school 


60.1 


9.0 


54.5 


staff having a say in fmancial decisions 


36.6 



In interpreting the tables the extent to which RSS was perceived to have affected an area needs to be 
considered alongside the extent to which that area was thought to have improved over the last three 
years. It is notable that the areas in which school have reported the most improvement (SDP 74%, 
Vision 92% and Staff development 73%) are also the areas in which RSS was reported to have had 
the most significant effect. Similarly relatively less improvement was reported in the areas in which 
RSS was said to have had the least effect (Morale 53.5% and collegiate decision making 37%). The 
exception to this last case is the leadership of the school in which 60 per cent of teachers reported on 
improvements but 36 per cent perceived no effect on leadership from RSS. For these teachers the 
appointment of a new Principal was seen as fundamental to improvement in the management of 
the school rather than the RSS initiative. The importance of headship is widely recognised as crucial 
to successful change in the school improvement literature. "The right sort of leadership is at the heart 
of effective schooling, and no evidence of effectiveness in a school with weak leadership has emerged 
from any of the reviews of research" (National Commission, 1996 p 335). 

The impact of new Principals cannot be attributed directly to RSS, nonetheless, improvements in the 
overall management at senior level were reported in most schools. In four schools (1 secondary and 
3 primary), in particular, RSS was said to have enabled various individuals to hone their leadership 
qualities. This included comments such as "the Senior Management Team ....has been allowed to 
develop" and Vice Principals were cited in particular for showing "strong management style and 
expertise." 

An analysis of written comments showed that in general terms teachers from eleven of the 14 schools 
thought RSS had helped in sharpening the focus of management as it had encouraged them to "look 
more closely at the running of the school" which had implications for school improvement. 



Areas in which RSS appears to have had the most impact 

School development planning 

Nearly three quarters (74 %) of teachers thought SDP had improved during the period of RSS with 
40 per cent crediting RSS as strongly instrumental in this. Teachers reported that the emphasis on 
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planning (APs) had "given direction for overall school development" which had also contributed to 
the "development of schemes of work." This conclusion is in line with Advisors' perceptions on this 
improvement in capacity to plan, engendered by RSS reported earlier. 

There was evidence of considerable variation between schools in teachers' perceptions of 
improvement in management. In nine schools more than two thirds of respondents thought 
improvement had occurred over the last three years, although in only five schools was RSS seen to 
have a significant effect (by over half the sample) all of which were primaries. The range was from 
zero to 80 per cent. 

Developing a vision of what the school is trying to achieve 

The vast majority (92%) of teachers thought their school's vision had improved, by far the highest 
rating for any item. Over a third (38%) thought RSS had had a significant effect on this change. 
Teaches wrote that RSS had helped develop vision by making "staff more aware of (the) direction the 
school is moving because the "aims of the school" had been made clear. Teachers had been set clear 
"goals" and had a firmer notion of "what we want to achieve." 

Teachers in all schools reported improvement in the school's vision (for 12 schools this was noted by 
over 90% of respondents). Again for five schools (all primaries) RSS was seen to have had a significant 
effect in promoting this improvement. The range was from only seven to 83 per cent of teachers 
attributing a significant effect to RSS in their school. The small numbers of respondents in some 
schools (in two primaries and one secondary less than ten questionnaires were returned) means that 
differences between individual schools should be treated with great caution. 

Improvements in the staff development programme 

As with the two categories above, improvements in this area were also seen as significant. When 
teachers chose to write extra comments on the questionnaires thirty-six teachers from eleven schools 
chose to comment on this aspect of RSS. In written comments the schools' staff development 
programme was perceived to have been the area most affected by RSS. Whilst RSS enabled teachers 
to update their skills on INSET courses outside of school, teachers thought the opportunities for 
intra-school staff development of greater importance. In conclusion RSS was said to have, "enabled 
teachers to work together in the classroom and learn from each other in a non-judgemental situation" 
which "enabled staff members to share experiences and support each other." 

This perception concurs with the observations made on the importance of staff development 
programmes for school improvement by Barber and Dann (1996) who concluded that: "a successful 
urban school requires a learning staff" (p 22). 

In all, for seven schools improvement in the staff development programme was reported by more 
than two thirds of teachers (the average was 73%). However in only five schools (all primaries) was 
RSS seen to have had a significant effect on this. The range was from zero to 80 per cent. 

Improved communication 

Overall, improvements in communication between the Senior Management Team and the staff was 
reported by over two thirds of teachers (69.7%). However this was not always seen to have been as 
a direct result of RSS. The difference between those teachers who thought RSS had no effect on this 
(26.4%) and those who thought it had a significant effect (22.0%) was small. An example of where 
RSS had been influential was summed up by one teacher who said "RSS has 'opened' the management 
of the school in that our RSS co-ordinator has adopted a policy of informing the staff about decisions 
etc. Staff morale is much higher as we feel we are all working together." In some schools not only 
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was communication improved at an institutional level, where "the process of planning for RSS has 
focused SMT's minds on the issue of planning, communicating and working with staff," but also at 
a personal level. "We are able to spend quality time on a one to one basis with all the SMT." The 
general benefits reported in this area were "improved communications, staff being kept informed and 
made to feel part of a school team." By contrast, RSS was not perceived to have led to improvements 
in the area of communication in schools where the project was seen as a primarily senior management 
initiative, little shared with the rest of the staff. 



Areas in which RSS appears to have had the least impact 

The three areas of management in which RSS would appear to have had a relatively lower impact are 
staff morale (34.1% no effect), leadership (36.2% perceived as noted earlier) and staff involvement in 
financial decisions (54.5 % no effect). 

Staff morale 

Just over half the teachers in the survey (50.8%) thought staff morale had improved over the period 
of RSS. In some cases this was accounted for by factors outside of the scope of RSS, (34.1% thought 
RSS had no effect in this area). Where RSS was thought to have had no influence on morale, teachers 
reported difficulties in coping with the pace and nature of 'change' brought about by 'initiative' type 
developments. This was summed up by one teacher who wrote that s/he was experiencing "too many 
changes in too short a time. Bombarded with information and no time to have a quiet think about 
the implications." 

Where RSS was thought to improve teacher morale this appeared to be as a direct result of additional 
resourcing. Some teachers reported being "delighted" with the additional resources they had access 
to as a result of RSS and one teacher wrote that this had a beneficial effect on teachers' morale because 
there had been improvements in not only working conditions but "things I have needed for years I 
have now got." 

The morale of staff working in disadvantaged urban areas is an extremely important issue which need 
to be taken into consideration when discussing school improvement. For example, Maden and 
Hillman (1996) state that "schools serving disadvantaged and frequently troubled areas, an abundance 
of energy and commitment is needed just to tread water" (p 335). Schools in such areas often find it 
hard to recruit and attract staff and the stigma of being associated with a 'failing school', in raw league 
table terms may exacerbate recruitment difficulties. 

Staff having a say in financial decisions 

The questionnaire asked teachers nine questions about management issues in their school. Teachers 
reported the least improvement (36.6%) in this area and over half (54.5%) felt RSS had not affected 
their involvement in financial decisions. By contrast less than one in ten (9.0%) of teachers believed 
RSS to have had a significant effect on this area. Whilst Principals and SMT may have been 
increasingly willing to consult with teachers on other matters there still appears to be a reluctance in 
most schools to include teachers in financial decision making. Only five teachers from four schools 
chose to comment on this issue. These were from schools where they felt they had been involved in 
some financial decisions reporting that certain "staff involved had an increased say on how money 
is spent in their co-ordinating areas." 
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Other respondents 

The questionnaires provide a picture of teachers' views of the impact of the initiative on management. 
This question was also explored with other groups involved in the initiative. Two particular 
improvements were cited by CoGs as being relevant to management, as well as observing more 
structured staff development programmes, they were also of the opinion that "attitudes in general" 
were more conducive to school improvement as a result of RSS. 

For the majority of Principals and Co-ordinators, the opportunities RSS presented for senior 
management to improve their communication and leadership skills and focus on 'target setting' were 
seen as the key to improvements in management. Positive developments were reported such as a 
clearer focus on planning and an increased awareness of the importance of the School Development 
Plan coupled with a better approach to the development of whole school policies. Principals reported 
that RSS had made them much more aware of their teachers' staff development needs. 

The BELB Field Officers who worked with schools as part of the MBw RSS initiative echoed the 
findings of the CoGs and the Co-ordinators and added to be debate about management. They thought 
that RSS had been influential in highlighting the value of senior management monitoring what is 
happening in classrooms in a more systematic way. 



Crucial to the success of RSS in some schools was, as reported by BELB personnel, the extent to 
which the management structure within the school was sensitive to the need for improvement." It 
is evident that when respondents referred to 'management', they often meant the vision and 
commitment of individual Principals. The importance of the headteacher has already been 
commented on and in many respects RSS is no different to many other school improvement 
initiatives in that Principals demonstrated varying degrees of commitment and involvement. The 
relationship between the leadership of both senior and middle managers (heads of department) for 
secondary schools and school culture has been noted in studies of inner city schools in other contexts. 
(Sammons, Thomas and Mortimore 1997). A comment from Success Against the Odds echoes the 
sentiments of the BELB interviewees who drew attention to this areas: "at the heart of each school 
is the headteacher's vision and analysis of what constitutes good learning and teaching. The refusal 
to be deflected from this and its corollary - an understanding of the difference between means 
(intensive and sensitive work with parents, for example) and ends (high academic and social 
achievement ) - marks out these headteachers and their colleagues" (p 336). 



THE IMPACT ON TEACHING AND LEARNING 

The MBW RSS set out to tackle under achievement and raise school standards. Any attempt to 
improve pupil outcomes needs to focus on the quality of teaching and learning (Creemers, 
1994;Slavin, 1996; Reynolds, 1997). The teacher questionnaire contained a number of items which 
attempt to address the impact on classroom practice. 
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The tables for this section follow the pattern shown in the previous section on management. 



i areas in which teachers considered RSS had a significant effect 



significant 


no 




% improvement 


effect 


effect 






51.4 


11.0 


curriculum development 


88.6 


33.1 


25.4 


teachers collaborating to plan teaching 


81.1 


31.5 


14.4 


planning of lesson materials 


82.7 


30.4 


17.1 


quality of teaching generally 


80.0 


29.7 


22.5 


the quality of school policies 


75.2 


27.4 


16.2 


teachers focus on learning 


77.3 


26.9 


20.9 


the implementation of school policies 


78.5 



n= 189 



ii areas in which teachers considered RSS had least effect 



21.3 


31.5 


the frequency of discussion with SMT 


59.0 


16.2 


38.5 


the feedback given to pupils about their work 


59.2 


12.8 


48.3 


consistency in regular setting & marking homework 46.2 



n= 189 



In line with the figures reported on management it is striking that the areas in which schools have 
reported the most improvements (curriculum development 88.6%, teachers collaborating to plan 
teaching 81.1% and the planning of lesson materials 82.7%) are also the areas in which RSS was 
reported to have had a significant effect. Again, rather fewer respondents reported improvement in 
the areas in which RSS was said to have had no significant effect (SMT discussions 31.5% no effect, 
pupil feedback 38.5% no effect and homework 48.3% no effect) by a substantial minority. 



Areas in which RSS appears to have had the most impact 



Curriculum development 

Curriculum development was seen to have improved by nearly 90 per cent of teachers and over half 
felt RSS had played a significant part in this. Teachers reported that RSS enabled "curriculum 
development to have a greater time allocation" which resulted in "greater opportunity for curriculum 
development" and the ability to "target certain areas of the curriculum." Developments were 
reported at both institutional level because RSS "facilitated curriculum panels to function and allowed 
us to maximise the curriculum/ subject expertise present on the staff" and at an individual level where 
"teachers have been free. ..to follow up discussions and develop their ideas on curriculum delivery." 
Curriculum development in this context for many teachers was regarded as synonymous with staff 
development. Where improvements in teaching and learning occurred, particularly in the areas of 
language and maths, teachers reported that INSET opportunities, updating policies and the 
introduction of new programmes had brought about changes, "practical in-service training which 
included demonstration lessons by Field Officers has helped in the teaching of reading and practical 
mathematics." 



The majority of teachers in all schools (89.1) thought there had been improvements in curriculum 
development in their school over the last three years. Only in one primary school did less than two 
thirds of staff think this area had improved (60%). Amongst the secondary schools the range was 
from 67 per cent of teachers reporting improvement to 82 per cent. Only in six schools (all primaries) 
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was RSS seen to have had a significant effect on curriculum development by over half the staff who 
completed the questionnaire. 

Teacher collaboration 

Over 80 per cent of teachers reported being drawn together in schools to work on various aspects of 
RSS. The result of this increased collaboration appears to have been more collegiate pl ann ing and, in 
some cases, the development of new schemes of work. A common comment from teachers was 
"many areas of teacher collaborating and planning greatly improved by meetings with RSS appointed 
staff" and "better schemes of work and better forward planning have led to greater improvements 
in development of (the) curriculum and have contributed greatly to the overall teaching within the 
school." This emphasis on planning has felt to have led to great improvements in the quality of 
teaching and learning within many of the schools. 

As well as reporting greater collegiality in planning the content of lessons, RSS appears to have 
encouraged more thought to be given to resourcing activities. Where this happened teachers reported 
that the additional resources offered to the schools as a result of RSS made an important difference 
to the quality of teaching and learning in their classrooms. One teacher commented, "the availability 
of resources in various curricular areas has enhanced planning and widened the possibilities for 
children's learning experiences." On the practical delivery of lessons another noted, "purchase of 
added resources for English and mathematics have made the planning of lessons much easier." A 
number of schools chose to use RSS money to appoint media resources officers/technicians to assist 
teachers in preparing lesson materials. 

Improvements in the planning of lesson materials was perceived to have occurred in all but one school 
by a significant majority of teachers (the overall average was 83%). The range was from 40 to 100 per 
cent at the school level. Again, however, the percentage who indicated that RSS had had a significant 
effect on this varied markedly between schools from zero ( 3 schools) to 74 per cent. Only in five 
schools did half or more of staff think RSS had a significant effect. Again these were all primary 
schools. 

The quality of teaching generally and teachers 'focus on learning 

Teachers were asked about both the quality of teaching and about teachers' focus on learning. 
Teachers reported that there had been improvements in both areas (88% and 77% respectively). Also 
over a third indicated a significant effect of RSS (30% and 27% respectively) on the quality and on 
the focus of teaching and learning. The teachers from ten schools who chose to write about this areas 
thought the initiative had helped them to look more critically at classroom practice. The further 
comments teachers made split them into three distinct sub-categories. The largest group were teachers 
who thought the initiative had enabled them to consider the learning taking place in their own 
classrooms: "emphasis now on learning rather than teaching" and "teachers are encouraged to focus 
on pupils' learning and completion of tasks." The second, slightly smaller group were teachers who 
considered both teaching and learning, "RSS has allowed teachers to focus on their teaching and the 
children's learning and make significant changes." These teachers indicated an added depth to their 
work as a result of RSS, "the staff have come together to look at all aspects of teaching and learning 
more thoroughly." The third group, equal in size to the second group, placed the emphasis on 
teaching which was being done more thoughtfully: "teachers are putting more thought now into 
what they are teaching and standing back more and thinking about the best way to put over a topic" 
and "teachers more aware of what they deliver." 

Where changes in teaching and learning were reported teachers often put this down to the 
development of their own expertise arising from extra time to examine and develop their classroom 
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practice. As one teacher wrote, "discussions and training sessions and visits from people with 
expertise, reading documents, watching videos and keeping records. All these have caused us to 
re-examine our teaching methods, sometimes for the better." 

Views about improvements in the quality of teaching over the last three years were generally positive 
(over two thirds indicating this) in eleven of the 14 schools. The range was from 30 per cent to 100 
per cent (2 schools). Again however, only in five primaries was RSS seen to have had a significant 
effect. The range was from zero (2 schools) to 69 per cent. 

Views about teachers focus on learning revealed that most teachers believed improvements had 
occurred. The range was from 50 per cent to 100 per cent. However in eleven schools over two 
thirds reported improvement. Nonetheless in only two schools did more than half the sample 
attribute a significant effect to RSS (range zero to 67 per cent). 

Teachers varied in their perceptions of improvements in the quality of schools' policies. In eight 
schools more than two-thirds of teachers believe improvement had occurred, but only in three (all 
primaries) did half or more think RSS had had a significant effect. 



Areas in which RSS appears to have had the least impact 



The feedback given to pupils about their work 

The importance of appropriate teacher feedback on work has been identified in reviews of school 
effectiveness research (Sammons, Hillman, Mortimore 1995). Research by Gipps and Tunstall (1996) 
has likewise highlighted this aspect. Although 'feedback' was not targeted specifically by RSS it is an 
important aspect of pedagogy. Fifty-nine percent of teachers reported improvements in teacher 
feedback to pupils about their work, although relatively few attributed RSS as having a significant 
effect (16%). It is possible that teachers did not regard this areas as important in determining the 
quality of teaching and learning and /or that the focus of RSS development work did not give a 
priority to this area. 

It should be noted that five teachers from four schools said the teaching and learning in their school 
was good before RSS, with good planning and policies which had been implemented in some of the 
target areas prior to their inclusion in RSS. 

Consistency in regular setting and marking homework 

Just under half (46%) of teachers thought that this area had improved in the last three years and only 
a s mal l minority (12 per cent) thought RSS had had a significant impact. There is a strong tradition 
in Northern Ireland schools (both primary and secondary) of homework being set on a regular basis. 
Whilst there has been little directly reported differences in the teachers attitudes to homework, there 
is evidence to suggest that parental attitudes have improved, with parents being more involved in 
'learning at home schemes', especially in reading (see impact on parental involvement). The APs do 
not suggest that schools saw homework as a focus of RSS and the teachers' responses support this 
conclusion. 

The pace of change 

Six teachers chose to comment specifically on how RSS had altered the pace of existing improvements 
in teaching and learning, "factors were already improving before RSS but it quickened the process and 



kept the improvements going" and "there was a noticeable improvement in standards before RSS, but 
RSS certainly acted as a catalyst for change and speeded things up." Again it must be remembered that 
many schools had been involved in other development projects prior to RSS and indeed in reading 
in particular the paired reading strategy was a focus of earlier development work. 

Other respondents 

Whilst most of the above has focused on teachers' views of the impact of the initiative on teaching 
and learning this question was also explored in interviews with other groups involved in the initiative. 



Over half of the Principals claimed the initiative had enabled them to emphasis the importance of 
teaching and learning with their staff. Some Principals also reported an improvement in their 
teachers' subject knowledge, in particular IT skills. The most important impact RSS made on 
teaching and learning reported by Principals, however was changes in teachers' attitudes ("they lost 
some of their cynicism", "they began to look beyond their own classrooms", "staff see themselves 
as learners") and practices (a focus on how to teach as opposed to what to teach) which went beyond 
"cosmetic changes" and resulted in greater consistency in delivering the curriculum. The RSS 
experience also made Principals increasingly aware "that teachers need to reflect on their own 
teaching and learning." RSS was credited by some Principals for giving flexibility to allow teachers 
time in which to "review their classroom practice." 

All Co-ordinators agreed with Principals on the positive impact RSS had on staff development and 
changes to teachers’ attitudes and practices. Co-ordinators considered that in addition, RSS had 
helped them to raise teachers' expectations of pupil performance which had implications for teaching 
and learning. 

Field Officers identified a number of schools where teaching and learning, in their opinion, had 
improved as a direct result of RSS. In these schools teachers had been given opportunities to discuss 
"good practice" and to examine their "core values" and how this contributed to "effective" teaching 
and learning in their school. 

THE IMPACT ON EDUCATIONAL STANDARDS 



he MBW RSS initiative was specifically instituted to improve levels of pupil performance in reading 
and numeracy in schools identified as under-achieving. This section will consider changes in pupil 
performance and access to Higher Education and Further Education from questionnaire, interview 
and schools' performance data. 



areas in which teachers considered RSS had a 



significant effect 



significant 


no 




% improvement 


effect 


effect 






47.5 


5.6 


Standards of reading 


88.5 


36.8 


8.6 


Standards of numeracy 


81.5 


30.9 


21.3 


Teachers’ goal/target setting for pupils 


77.1 


29.2 


12.9 


Quality of learning in the school 


82.4 


26.1 


21.6 


Teachers’ focus on pupil outcomes/results 


75.9 



n= 189 
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areas in which teachers considered RSS had least effect. 



significant 


no 




% improvement 


effect 


effect 






21.9 


23.8 


standard of academic achievement for at Y7 (prim only) 


71.8 


13.2 


29.1 


academic results 


66.6 


23.7 


29.4 


Consistency in standards between teachers 


63.4 


13.7 


34.2 


end of key stage assessments 


57.1 


14.4 


37.4 


preparation of students for exams end or end of KS assessment 


57.4 


8.2 


38.4 


standard of academic achievement at Y8 (sec only) 


63.4 


1.8 


58.9 


vocational results(sec only) 


32.7 


3.4 


66.1 


number of pupils going on to HE/FE 


15.0 



The impact of RSS on educational standards as measured by the teachers' questionnaire shows some 
marked similarities and differences to the two previous sections on management and teaching and 
learning. It is similar because the area seen as most improved (reading 86 %) is also the area in which 
RSS was reported to have had the most significant effect (48 %) whereas the area viewed as least 
improved (number of pupils going on to Higher or Further Education 15%) was also that in which 
RSS was seen to have had no effect by two thirds of respondents( 66%). 

In the light of the aims of the MBW RSS it is particularly encouraging that the vast majority of 
teachers perceived that improvement in standards of literacy and numeracy and in the quality of 
learning in the school had occurred over the preceding three years. By contrast the two areas in 
which only a minority felt improvements had been made concerned vocational results and the 
number of pupils going on to higher and further education (secondary schools only) may indicate that 
these outcomes had received relatively less attention than others. Secondary schools on the whole 
targeted Year 8 pupils (in 1994/5) who will not enter public examinations until 1998/9 onwards. The 
effect of MBW RSS on examination results would not therefore be evident until 1998/9. Examination 
results and entry into FE/HE are both areas which could be more susceptible to outside school 
influences (including HE and FE opportunities, parental and community expectations and local 
employment prospects). There was some variation evident in teachers' views about improvement and 
the impact of RSS between the 14 schools. The small numbers of respondents in some schools (in two 
primaries and one secondary less than ten questionnaires were returned) means that differences 
between individual schools should be treated with great caution. 

Secondary School Examination Results 1993/4 - 1996/7 

The four secondary schools' examination results were analysed to establish whether any trends could 
be discerned over the four year period 1993/ 4 - 1996/7. As noted earlier in the report schools had 
focused their efforts on Year 8 entry cohorts giving priority to literacy and numeracy. It is therefore 
too early to trace any impact on these pupils' performance at GCSE. Nevertheless, it is disappointing 
that in terms of published examination results, there was limited evidence of any improvement. 
None of the secondary schools showed any noticeable increase in the percentage of pupils gaining 5 
GCSE passes (Grade A-C). In terms of the measure 1-4 GCSE (Grade A - C) there was also no 
evidence of improvement. Using a broader measure (% 5 GCSE passes A-G) there was slight evidence 
of improvement for two schools for the percentage gaining five A-G passes. In terms of no passes 
at GCSE or other examinations there was also some evidence of improvement . 

There was also evidence of improvement in the percentage of pupils obtaining one or more passes in 
non-GCSE examinations for three out of the four econdary schools schools . 
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Reading and Numeracy across all schools 



The majority of teachers (over three quarters) in 12 schools believed reading and educational standards 
had improved over the last three years. Only in two schools was the trend weaker (60 and 63% 
respectively citing improvement). For numeracy a similar trend was evident although in two 
secondaries and one primary less than two thirds thought standards had improved. An analysis of 
schools APs indicated that, during the first year of the initiative, all schools focused on reading with 
numeracy receiving more attention in the second or third year. 

The RSS initiative was seen to have had a significant effect in raising reading standards by over half 
of teachers in five schools (two secondary, three primaries). The range was from zero to 89 per cent 
attributing a significant impact on reading. For numeracy RSS was perceived to have had a significant 
impact in four schools (one secondary, three primaries). The range was from zero (two schools) to 
89 per cent. 

With respect to target setting for pupils, teachers' views again varied at the school level. Although 
the majority reported improvement (over two thirds in 12 schools) the range was from 11 to 100 per 
cent. Only in three schools, however, did over half the teachers attribute a significant effect to RSS 
(one secondary and two primaries). The range was from zero (3 schools) to 67 per cent (2 schools). 
For the quality of learning improvement was cited by 12 schools the range being from 1 1 per cent in 
one institution to 100 per cent (5 schools). Again RSS was perceived to have had a significant impact 
by half the staff or more in four schools (all primaries). The range was from zero (five schools) to 67 
per cent. Improvements in the extent to which teachers focus on improving pupil outcomes was 
noted by over two thirds of staff in 12 schools. The range was from 65 per cent (two schools) to 100 
per cent (two schools). Nonetheless, only in three schools (two primaries) did more than half the staff 
think RSS had had a significant effect on this area. The range reporting a significant effect was from 
zero (three schools) to 68 per cent. 

Areas in which RSS appears to have had the most impact 

Improvements in standards of reading and numeracy 

Teachers generally had very positive views about the extent of improvement over the last three years. 
Nonetheless, only a minority attributed a significant effect to RSS. The interview data, however, 
indicated unanimous agreement between Principals, Co-ordinators, CoGs and BELB personnel that 
RSS had improved standards in reading and general numeracy across all schools. Several reasons were 
given for the improvements in reading which included the introduction of : 

• additional reading time on the timetable 

• Reading Recovery programmes 

• new approaches to reading such as 'paired/peer reading' and 'Reading Clubs' 

• the purchase of new published commercial schemes 

• new library facilities and reading/library clubs 

• a special focus on one curriculum area by all staff leading to consistency 

• new equipment and book provision (which was reported to have motivated children who 
were "displaying enthusiasm for and enjoyment of reading" unseen before RSS). 

Numeracy was seen to have benefited mainly because of the additional resources provided by RSS. 
This was said to have affected the work of teachers by extending their teaching repertoire, most 
notably in the areas of practical work: "extra equipment has been acquired enabling classes to benefit 
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from practical activities." Test scores were also cited in support of the assertions of improvement in 
reading and numeracy. 

Whilst improvements were noted in connection with both of these core curriculum areas the 
emphasis in the first year of RSS for many schools was literacy with the focus on numeracy coming 
later on and, as noted later, test score evidence for numeracy was less widely available. 

The evidence from school effectiveness research that reading performance is a crucial component of 
later academic performance (Sammons et al 1995) also indicates that socio-economic disadvantage has 
a greater influence on attainment and progress in this areas than in mathematics (Mortimore et al 
1988, Brandsma and Knuver 1989, Sammons et al 1993). Given its importance for access to the 
curriculum, the focus on reading during the project's first year can be seen as an appropriate top 
priority for schools. 



Teachers' focus on pupil outcomes /results 

Where improvements were reported in this area teachers said RSS had helped their school give greater 
consideration to measured standards because of better tracking systems for monitoring progress, 
"focus on outcomes and target setting has significantly improved percentage of top grades in the 11 + 
exam over the last three years" and "teachers are focusing on what the pupils should be able to do at 
the end of teaching unit, RSS has given time to think about this." In one school teachers reported 
being more aware of outcomes because of the introduction of a computerised learning system (ILS). 
However in four schools teachers reported that new assessment arrangements introduced before RSS 
had affected teachers' approaches in this area and therefore did not see RSS as the main catalyst for 
change. Many Principals and Co-ordinators indicated that RSS had made them more aware of pupil 
outcomes and performance scores and the importance of monitoring these. One secondary school 
reported the introduction of a new programme of regular tests for specific age groups as a result of 
discussions about pupil outcomes and this was a consequence of their involvement in MBW RSS. 

The MBW RSS schools varied in the approaches they used to monitor the impact of their initiatives 
on pupils' educational outcomes. As illustrated in the section on Action Planning, schools' varied 
in their specific objectives, the pupils or year group targeted and the kinds of evidence collected. 
Because of this it is not possible to make systematic comparisons across schools of, for example, rates 
of pupil progress over the period of the initiative. By contrast, the expanded RSS (RSSI 1995) has used 
a value added framework, with common baseline and post test measures to enable such analyses to 
occur. 



Areas in which RSS appears to have had the least impact 

Whilst teachers thought there had been general improvements in many areas they appeared reluctant 
to ascribe this as a significant effect of RSS because, "the programme has not really been in effect long 
enough to calculate improvements to standards through academic results" or as another teacher put 
it, "it is too early to see any significant change in end of Key Stage assessments, change in this area 
might be observed in two or three years time." Principals were able to cite improvements in specific 
areas but were cautious in making claims for sustained improvements. All four secondary schools 
thought it too early to see any impact on rates of entry to HE and FE and for other vocational and 
academic results. 
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Given the emphasis paid to the intake year (Year 8) in secondary schools' Action Plans it is clear that 
a three year evaluation cannot pick up any longer term effects. Follow up at age 16 would be needed 
to examine improvements in public examination and FE participation. 



THE LEGACY OF RSS AND CHALLENGES FOR THE FUTURE 

In any developmental project such as RSS which has a fixed time scale and is based on additional 
funding, the inevitable question of "what happens when the project is over and the funding runs out? 
is bound to be asked. A number of respondents reported on what they perceived as challenges for 
the future. 

The BELB's perspective 

In many ways, BELB personnel considered the initiative as an amalgam of "strategies for future 
issues , in that it introduced both schools, Officers and Advisors to some of the elements important 
in effective long term school improvement planning. It asked schools to identify priorities, and set 
targets through the drawing up of APs. It introduced them to alternative methods of monitoring, 
evaluating, staff development and self-evaluation. Concerns for the legacy of the initiative were 
reported for those schools which were perceived not to have taken "ownership" of the aims of the 
initiative. These were schools where staff were perceived to have failed to see the school improvement 
potential of the initiative. They continued to focus on RSS as a simple curriculum focused project and 
saw the additional resources being the key to success. It was feared that, for these schools, the 
withdrawal of additional resources will mean in effect that work on school improvement, in this 
particular format, comes to an end. There was also a concern expressed that it would have been 
helpful to schools if exit strategies had been "built into the project from the beginning", it wasn't 
good enough to flag these up during the last year when the end was in sight. 

Challenges for the BELB itself, in continuing the work started on this initiative were also cited. If 
other schools are to benefit from the lessons learnt during RSS it was said that the support services 
must have the "philosophy and framework in place for taking on school improvement." Respondents 
felt that BELB needed to look towards not only how it would continue to "skill" schools in school 
improvement, but also how it trains its own teams. It was argued that BELB itself needed to ensure 
that the focus for schools is not on resources but the way these may be best used to enhance the 
quality of teaching and learning. 

The Field Officers' perspective 

Across the range of "successful" RSS schools FOs considered the following a legacy of the project:- 

• teachers having been given the opportunity to discuss "good practice" and to examine their "core 
values" were now able to think about factors which contribute to "effectiveness" in schools 

• greater collegiality, more whole school planning with teachers feeling less isolated 

• more sophisticated Action Planning and target setting 

• better primary/secondary links and following on from this improvements in transition 
arrangements 

• better managed staff development programmes tied to development planning 

• a recognition of what can be achieved when teachers are "empowered" 

• the importance of the senior management monitoring what is happening in classrooms 

• an increased awareness of pupil outcome information 



The Chairs of Governors' perspective 

Seven CoGs thought RSS would leave a lasting legacy in their school. Five of these Governing Bodies 
were reported to be considering their LMS budgets with a view to supporting work started by RSS 
i.e. Reading Recovery programmes, class release for Co-ordinator etc. They wanted to maintain the 
"uplift to carry teachers forward" which RSS had provided. The two other schools, how had little 
flexibility in their LMS budget to support development directly were, nevertheless, keen to continue 
the work of RSS by creating opportunities for further staff development, especially in the core 
curriculum areas. 

By contrast four CoGs thought there would be little future for RSS in their school beyond the official end of 
the initiative due to the lack of continuing financial assistance and staffing difficulties (both loss of 
teachers and senior management conflict). Oneb felt that an amalgamation had overtaken the school and was not 
ure how RSS would be built on in future. 

The Principals' perspective 

Losing additional staff was the biggest concern for Principals (eleven out of thirteen interviewed) in 
continuing the work of the initiative. All of these said they were going to look critically at their LMS 
budgets to see if they could retain some of their RSS staff but for many this would be extremely 
difficult. Other concerns reported by Principals were the funding staff development and getting staff 
through the implementation "dip" once the "spotlight" of the Inspections was removed. 

The Co-ordinators’ perspective 

In five schools the Co-ordinators considered that budgetary constraints and the loss of additional 
staffing would have serious implications for the continuation of the work started on RSS. Similarly, 
the return of the Co-ordinator to the classroom was said to mean a period of consolidation rather 
than progress. Six Co-ordinators who reported success in the area of language (reading) said they 
intend to promote the RSS model of development in other curriculum areas, especially numeracy. 

Nine Co-ordinators said they intended to continue with the work of the initiative in the areas of: 

• continuing to develop the schools' reading provision 

• the management of difficult/ challenging children, through withdrawal provision or merit systems 

• redrafting policies and schemes of work 

• new year group programmes 

• improvements in curriculum planning 

• staff development 

• primary / secondary liaison 

• parental involvement and 

• developing monitoring and evaluation processes. 



The Teachers’ perspective 

Teachers' comments revealed five areas which could be considered the legacy of RSS in their school. 
All percentage are of those who chose to write comment on the questionnaire. 



• Working together 

For example one wrote, " if teachers like pupils are motivated sufficiently their goals rise and they 
achieve more" and another commented "I feel that we have learnt that if you have a good team of 
teachers and a clear vision of where you are going RSS can help you achieve your goals. " 
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• The issues of resources 

Some teachers said that resources were not always essential to improvements, "resources can help but 
the best resource is teachers". 

• The importance of teaching and learning 

One of the strengths of RSS specifically reported by teachers was it re-focused them on their primary 
job of teaching and forced them to consider contexts for learning. One teacher wrote, "we have learnt 
a considerable amount about our teaching and the attitudes to learning in the school" and "however 
difficult we must stop and think about what we're doing and why." 

• The importance of planning 

Some teachers drew attention to the importance of planning both for special projects, "an initiative 
of this magnitude needs careful planning," and for school development. When considering school 
development they highlighted the benefits of having a School Development Plan, "RSS has shown 

new ways to tackle old problems it has helped to concentrate ideas on forward p lanni ng i.e. The 

School Development Plan" and thus enhanced the school's capacity to develop and implement such 
plans. 

• Managing Change 

Teachers who wrote about change thought RSS had helped staff in understanding "how to undertake 
major change for sound educational reasons," recognising that "teachers have the power to change 
things when given the funding and resources needed." However, the view was also expressed that "it 
is impossible to make sweeping changes overnight." 

• Funding needs to be “fit for purpose”. Long term gains cannot be gauged by short term measures. 
Funders need to be clear in their initial aims and project outline how much work can reasonably be 
accomplished in the time scale they present. Unless this is done there is a danger of being over-optimistic 
with resultant disappointment. 

• Wherever possible financial allocations should be phased so that they fit in with schools natural 
methods of working. Where this is not always possible, schools should to be given reasonable notice of 
the financial cycle and more importantly they need to have the implications of the cycle made very clear 
to them. If specific deadlines have to be met e.g. in spending allocations , staff need to know the 
importance of these deadlines at the outset of the project to avoid rushed attempts to send allocations. 



The legacy of RSS must be viewed in the context of the many challenges experienced by UK schools 
during the 1990s e.g. introduction of the National Curriculum and national assessments, publication 
of league tables, LMS, as well as internal changes in individual schools (amalgamations, new Principals 
appointed etc.). 



Pupil outcomes 

In some schools it appears that RSS encouraged a greater focus on monitoring pupil outcomes and the 
adoption of specific programmes to raise achievement of either targeted pupils with very low prior 
attainment in reading and mathematics, or whole year groups. The extent to which gains in reading 
and mathematics ages are sustained over the longer term cannot be established as yet, however, 
indicators will be provided by KS3 and GCSE results for these cohorts as they progress through 
school over the next four years. Having said this, improvements in these basis skills should improve 
these pupils' ability to access the secondary school curriculum, and may help to improve self-esteem, 
motivation and attendance. 
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School Culture 

There was evidence in some schools that significant improvements in curriculum planning and the 
quality of teaching and learning had been fostered by involvement in the MBW RSS. Im provements 
in the quality of education are of course hard to measure in any straightforward way but the views 
of different groups of respondents provided valuable sources of evidence. The emphasis on staff 
development facilitated by extra staffing and work with Field Officers and Advisors also enhanced 
the 14 schools' capacity to plan and cope with change. The extent to which good practice can be 
transferred within schools and the stability of staffing who have gained new skills will of course 
influence the extent to which schools reap long term benefits from this extra investment in staff 
development. 

Cultural change is often seen as a key to school improvement (e.g. Fullan,1991; Stoll and Fink, 1992; 
Hopkins et al, 1994; Reynolds, 1995; Sammons et al; 1997). Change in the pupil and local community 
culture is also relevant. The need to foster high expectations and a belief that all pupils can learn has 
been seen to be important in the context of highly disadvantaged inner city schools. The experience 
of MBW RSS had a positive impact on teachers' views in many project schools. In some schools 
parental involvement had also been tackled although success in this was not always attributed directly 
to the project. 

Leadership 

The importance of the Principal and Senior Management Team's commitment to the MBW RSS was 
found to be crucial to its overall success in all schools. Where there were long standing management 
problems the capacity of schools and Co-ordinators, in particular, to implement RSS was constrained. 
In schools which experienced a change of Principal this was often viewed as an important catalyst for 
improvement, and generally more influential than RSS itself. 



CONCLUSIONS 

In evaluating the overall achievements of the MBW RSS initiative, it is crucial to recognise the nature 
of the particular socio-economic and political context in which the 14 project schools are set. If the 
challenges for education in the inner city are many, then the particular problems which face Belfast's 
schools can only multiple these challenges. The initiative sought to tackle under achievement in 
schools whose communities have had a long history of socio-economic disadvantage and in a highly 
selective system with a history of civil unrest. This initiative recognised that improvements do not 
necessarily come cheap and the funders are to be congratulated for investing in this work over a 
significant three year period. Nonetheless, the true benefits of the MBW RSS may not as yet be fully 
apparent. It must be recognised that in 'school improvement' there are "no quick fixes" (Gray & 
Wilcox, 1995; Stoll and Myers 1997) and that the seed sown today may take some time to come to 
harvest. Given the rushed and problematic start to the project in 1994 much of the first year was 
taken up with planning and in some schools it was not until the second year that developments really 
took off. Moreover, the focus on younger age groups in the secondary schools means that any long 
term impact on public examination results and HE/FE participation will not be picked up until 1999 
onwards. 



Management 

The initiative was conceived with clear and ambitious aims and addressed a real need to focus on the 
difficulties faced by under performing secondary schools in particular. Given the nature of 
educational funding it was right that the opportunity to use significant central funding for the 
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development of schools was taken. However, the lack of a realistic planning period to consider the 
nature of school improvement and the question of the most appropriate methods for tackling 
under-achievement meant that the start of the project was problematic. In an Authority with a 
history of innovative developmental projects an audit of past strengths and weaknesses associated with 
both management and development would have greatly assisted the MBW RSS in the initial phases. 



The new monitoring and evaluation strategies which MBW adopted during the period of the initiative 
meant they became increasingly familiar with developments taking place on RSS. Similarly, their 
direct involvement with BELB and school personnel, through the regular update meetings, helped 
to reassure them, as funders, that the initiative was being effectively and efficiently managed. It is 
important that funders are represented on relevant management bodies and that they ensure that all 
initiatives are auditing/ evaluated so that strenghts can be built on and any weaknesses addressed. 

The significant workload issues associated with managing the initiative were overcome and by the 
middle of year two, as the initiative progressed, the management became firmer. Personnel changes 
which occurred during the period of the work were dealt with effectively. There were many benefits 
for non-school based personnel (Advisors and Field Officers) from working on the RSS initiative. 
Involvement in the initiative has resulted in a greater understanding of the issues associated with 
school development and improvement which should be of great benefit for future projects. Much of 
the learning which has occurred on this initiative is already assisting in planning and developing the 
on-going DENI expanded RSSI which began in 1995. 



Launching the initiative and selection of schools 

The public naming of schools by newspapers as "under-achievers" soured the launch of the initiative 
in the secondary schools because schools appeared unprepared for the publicity this attracted. There 
is a real need to ensure that the approaches made to schools to become involved in such a wide 
ranging initiative are uniform. Many of the problems associated with the launch were overcome, but 
this delayed commitment to the project in some schools. The relative merits of so called 'naming 
and shaming' approach to school improvement remains of course controversial elsewhere in the UK 
(Mortimore and Whitty 1997). 

The selection of schools in both phases was controversial. Despite clear criteria there was 
misunderstanding and suspicion from both schools which were included as well as those excluded. 
These problems were also overcome but made for a difficult period at the beginning of the initiative 
and meant that progress in the first year was limited in many ways. 

Action Planning 

The schools involved in MBW RSS were not familiar in the initial stages of the initiative with the 
process of Action Planning. Schools struggled with trying to produce an Action Plan without, in 
many cases, sufficient training and support to tackle this complex operation. Where insufficient 
expertise was coupled with a time-table, perceived as unrealistic, and what appeared to be arbitrary 
decisions made about funding, many Principals in particularly, became jaded with the Action 
Planning process. Within the first year it was recognised by the CMC that Action Planning was an 
difficult issue for schools and this was addressed directly. From this point onward the initiative 
gained more credibility both in and outside schools and greater progress was evident subsequently. 
Schools and BELB staff made great gains in planning, targeting and implementing APs during the 





second year and the enhanced capacity to plan and manage change in schools was perceived by many 
involved to be an enduring legacy of the initiative. 

Monitoring and Evaluation 

Whilst the procedures put in place for monitoring the initiative were adequate these could have been 
much improved if those involved with making the system work had been involved in the planning 
phase. In this way a more realistic time-table could have been set and the duplication of information 
avoided. Although monitoring was consistent outside schools, opportunities for evaluating the 
progress made in schools was often hampered by a lack of comparable baseline measures. Where 
such measures were in place they tended to be in areas which could be tackled by standardised tests. 
The approach schools took to evaluation improved markedly as the initiative progressed, this 
appeared to be related to the ability to set clearer and more realistic targets in Action Plans. Because 
schools focused on different groups of pupils and did not adopt a common approach to baseline 
assessment of pupils and later follow up testing, it has not been possible to make direct comparisons 
of rates of progress across all schools. As noted in the original evaluation proposal, the nature and 
availability of any information on pupil attainments from individual schools affects the extent to 
which the evaluation can address the issue of pupil achievement gains over the three years. 

In some schools evidence of considerable short term gains in pupils' reading ages was submitted. If 
sustained in future years these gains should have a very beneficial impact on these pupils' subsequent 
progress at school and therefore on their long term educational outcomes. The adverse impact of 
poor literacy skills on young people's future ability to function in various aspects of adult life has 
been well documented by bodies such as The Basic Skills Unit. Although the APs in most schools 
emphasised reading, numeracy received much less attention in the first year. However, by the second 
and third year many schools had started baseline screening in mathematics and there was some 
evidence of attempts to transfer the success and achievements in focusing on reading into this core 
curriculum area. 

Support for MBW RSS 

Schools generally much appreciated the advice, help and support given to them as part of the MBW 
RSS initiative. The strategic use of Field Officers (FOs), in some schools, was particularly helpful as 
they coupled classroom credibility with curriculum expertise. There is evidence which suggests that 
many of the significant achievements made on the project could not have happened without this. 
It could be argued however that further developments, particularly during the initial stages of the 
project, could have been achieved had there been initial training for Advisors, FOs and Co-ordinators 
in issues associated with school improvement and school effectiveness for those taxed with the job of 
leading developments with and in schools. 



The role of DENI in MBW RSS 

DENI had an important role in RSS being influential in setting up, planning, managing and 
evaluating. Compromises had to be made to accommodate each of these different functions. Some 
schools found the process of inspection stressful, nonetheless there was clear evidence that schools 
which had the benefit of inspection evidence at the start of the project they were able to develop more 
realistic and meaningful Action Plans. The whole notion of 'improvement through inspection' could 
have been strengthened, if the criteria for subsequent 'evaluation' inspection (conducted after the first 
year) as opposed to a more formal inspection could have been made apparent and if there had been 
greater consistency in the verbal and written feedback schools received. Where the issue of 
management proved a barrier to school improvement there was a perception that this was not always 
addressed explicitly. 



The Impact of MBW RSS in Relation to the Initiative’s Foci 

There is clear evidence that the majority of schools involved in this initiative made improvements 
related to the main aims of the initiative. Nonetheless, there was variation between schools and 
within schools in the extent to which the school improvement' message had become embedded in 
practice. In schools that used the initiative to foster a whole school approach to developments, 
significant improvements were made, often transforming areas of practice. In schools where the 
Principals and Co-ordinator were less successful in involving all of the staff, little other than the 
benefits of three years of additional resources may be the legacy of the project. This evaluation was 
commissioned to look at a 'school improvement initiative' - not individual schools. The 
encouragement of principals and their staff to consider critically how the MBW RSS experience 
relates to their own institution and staff and their approach to school development may enable others 
to learn from the initiative’s legacy. 

As noted earlier, the lack of common baseline measures has made the evaluation of literacy and 
numeracy problematic. Pupil outcomes in the literacy and numeracy foci were more easily assessed 
for schools which had baseline data, but this was not the case for all schools. Few schools had 
rigorous methods for establishing baseline data outside of literacy and numeracy. Although schools 
were able to produce information on attendance, suspensions, exclusions and other statistical 
measures, there was only limited evidence in their paperwork to suggest that this information was 
interrogated or analysed to set targets and aid the development of consistency in policy and practice. 

Impact - Management 

One of the major success of MBW RSS was that it broadened the focus of many other initiatives that 
had gone before to include a 'whole school approach' to development. It provided a vehicle for staff 
to articulate a 'vision' for the school and to identify the steps that need to be taken if the 'vision' is 
to be turned into a reality. It appeared that many senior management teams were able to focus 
specifically on teaching and learning shifting the emphasis in some schools away from a management 
dominated by administration. There was evidence to suggest that the improvements in staff 
development programmes, linked to better long, medium and short term planning will foster further 
developments in the future. Clearly the stability of staff who received additional help and the extent 
to which they are able to share this with other colleagues will also have an impact on the extent of 
long term cultural change in individual schools. 

Impact - Teaching and Learning 

The MBW RSS enabled schools to develop more systematic and relevant staff development 
programmes which have fed into curriculum improvements. It provided time for key personnel to 
consider both subject knowledge and approaches to pedagogy. Linked to these developments has been 
a greater emphasis on collegiate planning which has had practical classroom application. The quality 
of lesson material has been improved and the awareness of the importance of a more appropriate 
curriculum, particularly at secondary level. It was clear that improvements in this area have been 
made in most schools. Nonetheless, it must be stressed that some (particularly the secondary schools 
which were operating from very low baselines) still have considerable scope for further 
improvements. The focus RSS gave to teaching and its links with learning, coupled with an emphasis 
on measuring pupil outcomes could be extremely important to future developments in man y of these 
schools. 



The MBW project has provided a considerable attempt to develop the improvement capacity of 
schools. There is evidence (Bollen 1989, Hopkins et al, 1994), that school improvement is related as 
much to the quality of the policy (raising standards) as the improvement capacity of schools. The 
school improvement literature suggests that, given greater school autonomy in many systems in 
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recent years, the concept of the thinking ( self-reflecting) school has to be promoted. The evaluation 
suggests that the MBW RSS initiative did indeed stimulate refeclection for staff, both individuals, and 
at an institutional level. It sharpened the measurement, monitoring, target setting and evaluating 
capacity of schools through the Action Planning process. In several schools significant curriculum 
development appears to have occurred which should have benefits over the longer term. 

Impact - Educational Standards 

Without standard baselines measures it is difficult to be precise about improvements made across the 
board on this initiative. There is evidence to suggest that for individual children in some schools 
there have been considerable gains from being involved in RSS, especially for poor readers. The 
evidence for gains in mathematics however, is weaker. How much these short term gains can be 
sustained is difficult to predict, but it appears that, particularly at secondary school level, pupils are 
now better able to access the N.I. National Curriculum as a result of their experiences on MBW RSS. 
It is too early to make assertions about the impact of RSS on public examination results (GCSE and 
Vocational) and access to HE and FE. Improvements in primary pupil outcomes should mean higher 
baselines for Year 8 on entry into secondary schools. Given knowledge of the links between reading 
performance at age 7 and public examination grades improvements in primary pupils' reading ages 
are likely to have a positive impact on their later examination results and employment prospects 
(Sammons 1995). In the long term the improvement of primary pupils' skills (for example related 
to the targets set by Literacy and Numeracy Task Forces) would do much to reduce the significant 
challenges faced by certain secondary schools where (as illustrated by some of the schools in MBW 
RSS) over 60 per cent of children may be two years or more below their chronological age in reading. 

Time scale 

Bollen (1989) argued that "as school improvement is a process and not an event, time is a major factor 
that cannot be manipulated without a strong influence on the quality of the process." He noted that 
innovation is often set on a three year scheme which can be too short for institutional change to be 
embedded or for the impact of change to be fully assessed. A five year programme may be more 
appropriate and it is likely that a follow up of pupil achievement, attendance and behaviour over the 
next two years by BELB would provide valuable evidence of any longer term gains in the 14 schools 
involved in the MBW RSS experience. 

The considerable difficulties faced by so called 'under-achieving 1 schools especially at the secondary 
level have been highlighted by inspectors and the media during the 1990s. The research evidence 
suggests that turning round such schools is extremely difficult (Gray and Wilcox 1995). The culture 
of such schools may be fragmented and focus on teaching and learning weak (Reynolds 1996). The 
MBW RSS initiative provided a major opportunity for schools to re-evaluate their approaches and a 
strong impetus for curriculum development. In most schools the capacity for planning was much 
improved during the three years and clear strategies for regular monitoring of reading was made in 
most schools, although the impact on numeracy to date appears weaker. In this context it is relevant 
to note that the need to develop specific approaches to develop pupils' basic number skills has been 
highlighted by the National Numeracy Project. This has recently reported substantial gains in pupil 
performance through structured interactive whole class approaches and it may be appropriate for the 
MBW RSS schools to consider ways in which such approaches could be adopted. The results of the 
National Numeracy Task Force may also proved valuable (DfEE 1998). There was some evidence 
of greater staff cohesion and clearer goals setting, although problems in senior management remain 
in some schools. The positive impact of a change of Principal was a noticeable feature in some schools 
and attributed with a greater impact in effecting improvements than the RSS initiative by teachers in 
these schools. The adverse impact of amalgamation on a school's ability to focus on the RSS initiative 
is also a relevant factor. 
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There is no doubt that the four secondary schools at the heart of the MBW RSS initiative continue 
to face considerable challenges in low levels of literacy and numeracy of their Year 8 intake. It is 
likely that this applies to several other secondaries serving socio-economically disadvantaged 
communities in Belfast. Ways of effectively targeting resources at the pupil level (without rewarding 
failing schools) to provide specific programmes to promote rapid gains across Years 8 and 9 to enable 
these pupils to access the NI curriculum may be needed if standards are to be raised in the long term. 

There may even be a case for temporary disapplication of the full national curriculum requirements 
in such schools to ensure a focus on basic skills for those with measured performance two or more 
years behind their chronological age. Intensive holiday clubs prior to secondary transfer may also be 
appropriate for pupils below a certain level although it is clear that raising achievement in the basic 
skills at the primary level is crucial. A number of RSS primaries had some success in these areas and 
ways of maintaining this emphasis and spreading good practice should be explored. The pupil culture 
in all boy secondary schools is also an important issue for staff seeking to promote better behaviour 
and discipline. It is hoped that the RSS secondary schools will share good practice in this area and 
continue to monitor trends in these outcomes. 

The MBW RSS initiative provides an important example of a school improvement initiative which 
had very clear and laudable aims focusing on promoting pupil outcomes and which combined 
considerable financial support with external advice and guidance in seeking to develop participating 
schools' capacity to improve. It thus attempted to integrate both a 'top down', external approach to 
improvement with the encouragement of 'bottom up' strategies developed within individual schools. 
The majority of those involved valued the opportunities the initiative provided and believed that 
much was learnt from the MBW RSS experience, both by individual schools and BELB personnel. 
As with any initiative, there were areas of success and aspects where less progress was made than 
anticipated. Schools developed their capacity to plan, monitor and evaluate school improvement, and 
the evidence suggests that the quality of teaching and learning was improved in many cases. 
Nonetheless, significant challenges remained in several schools, especially at the secondary level. 
Some positive effects on pupils' reading were identified, and attempts to transfer this to mathematics 
were being made. Behaviour, discipline and attendance remained areas of concern especially for the 
secondaries, although modest improvements in attendance were found in most schools. On the 
whole, secondary schools faced greater challenges and experienced more difficulties in implementing 
their improvement strategies than their feeder primaries. 

As yet the long term benefits of the MBW RSS are hard to judge, given the three year time scale. 
There is evidence that the experience of the project's first year provided a helpful input into the 
planning process for the expanded RSS province wide (Thomas & Sammons, 1996). A further follow 
up of pupil cohorts which were targeted in schools' Action Plans at Key Stages and in terms of public 
examinations and post-school destinations would be valuable to explore this issue in subsequent years. 



In the U.K. current work under the new Labour Government is attempting to develop ways of 
contexualising school performance and setting results in context to facilitate its policy of target setting 
and national strategies to raise standards. Moreover, the recently announced policy relating to 
Education Action Zones bears strong similarities to the strategy adopted in the MBW RSS initiative 
by focusing on low attaining secondary schools serving highly disadvantaged intakes and selected 
feeder primaries. This evaluation may therefore have implications which extend beyond the 
particular context of the 14 Belfast schools involved. 
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Appendix 1 



The following interviews were conducted with: 

6 BELB personnel including Officers and Advisors 
13 Principals 
13 Co-ordinators 
12 Chairs of Governors 
4 Field Officers 

Making Belfast Work - one group interview 

Teachers' views were sought both quantitative and qualitatively in the questionnaires, with a total 
of 1,650 written responses analysed. 

Questionnaire Returns 

The breakdown of returned questionnaires on which this analysis is based is as follows: 



Questionnaires 


Number of Returns 


% Rate 


All teachers 


189 


51% 


Teachers (Primary) 


124 


59% 


Teachers (Secondary) 


65 


41% 


All parents 


983 


57% 


Parents (Primary) 


764 


76% 


Parents (Secondary) 


219 


31% 


All pupils 


1268 


74% 


Pupils (Primary) 


918 


91% 


Pupils (Secondary) 


350 


49% 


Total 


2440 


64% 
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